The Mixtures of Experts model is a statistical method for classification and regression (Waterhouse, S., “Classification and Regression Using Mixtures of Experts, October 1997, Ph.D. Thesis, Cambridge University). Waterhouse discusses Mixtures of Experts models from a theoretical perspective and compares them with other models, such as, trees, switching regression models, modular networks. The first extension described in Waterhouse's thesis is a constructive algorithm for learning model architecture and parameters, which is inspired by recursive partitioning. The second extension described in Waterhouse's thesis uses Bayesian methods for learning the parameters of the model. These extensions are compared empirically with the standard Mixtures of Experts model and with other statistical models on small to medium sized data sets. Waterhouse also describes the application of the Mixtures of Experts framework to acoustic modeling within a large vocabulary speech recognition system.
The Mixtures of Experts model has been employed in protein secondary structure prediction (Barlow, T. W., Journal Of Molecular Graphics, 13(3), p. 175-183, 1995). In this method input data were clustered and used to train a series different networks. Application of a Hierarchical Mixtures of Experts to the prediction of protein secondary structure was shown to provide no advantages over a single network.
Mixtures of Experts algorithms have also been applied to the analysis of a variety of different kinds of data sets including the following: human motor systems (Ghahramani, Z. and Wolpert, D. M., Nature, 386(6623):392-395, 1997); and economic analysis (Hamilton, J. D. and Susmel, R., Journal of Econometrics, 64(1-2):307-333, 1994).